Goal-Conditioned Reinforcement Learning within a Human-Robot Disassembly Environment

The introduction of collaborative robots in industrial environments reinforces the need to provide these robots with better cognition to accomplish their tasks while fostering worker safety without entering into safety shutdowns that reduce workflow and production times. This paper presents a novel strategy that combines the execution of contact-rich tasks, namely disassembly, with real-time collision avoidance through machine learning for safe human-robot interaction. Specifically, a goal-conditioned reinforcement learning approach is proposed, in which the removal direction of a peg, of varying friction, tolerance, and orientation, is subject to the location of a human collaborator with respect to a 7-degree-of-freedom manipulator at each time step. For this purpose, the suitability of three state-of-the-art actor-critic algorithms is evaluated, and results from simulation and real-world experiments are presented. In reality, the policy’s deployment is achieved through a new scalable multi-control framework that allows a direct transfer of the control policy to the robot and reduces response times. The results show the effectiveness, generalization, and transferability of the proposed approach with two collaborative robots against static and dynamic obstacles, leveraging the set of available solutions in non-monotonic tasks to avoid a potential collision with the human worker

Robotic-Arm-Based Force Control by Deep Deterministic Policy Gradient in Neurosurgical Practice

This research continues the previous work “Robotic-Arm-Based Force Control in Neurosurgical Practice”. In that study, authors acquired an optimal control arm speed shape for neurological surgery which minimized a cost function that uses an adaptive scheme to determine the brain tissue force. At the end, the authors proposed the use of reinforcement learning, more specifically Deep Deterministic Policy Gradient (DDPG), to create an agent that could obtain the optimal solution through self-training. In this article, that proposal is carried out by creating an environment, agent (actor and critic), and reward function, that obtain a solution for our problem. However, we have drawn conclusions for potential future enhancements. Additionally, we analyzed the results and identified mistakes that can be improved upon in the future, such as exploring the use of varying desired distances of retraction to enhance training.The authors were supported by the government of the Basque Country through the research grant ELKARTEK KK-2023/00058 DEEPBASK (Creación de nuevos algoritmos de aprendizaje profundo aplicado a la industria). This study has also been conducted partially under the framework of the project ADA (Grants for R&D projects 2022 and supported by the European Regional Development Funds)

Robotic-Arm-Based Force Control in Neurosurgical Practice

This research proposes an optimal robotic arm speed shape in neurological surgery to minimise a cost functional that uses an adaptive scheme to determine the brain tissue force. Until now, there have been no studies or theories on the shape of the robotic arm speed in such a context. The authors have applied a robotic arm with optimal speed control in neurological surgery. The results of this research are as follows: In this article, the authors propose a control scheme that minimises a cost functional which depends on the position error, trajectory speed and brain tissue force. This work allowed us to achieve an optimal speed shape or trajectory to reduce brain retraction damage during surgery. The authors have reached two main conclusions. The first is that optimal control techniques are very well suited for robotic control of neurological surgery. The second conclusion is that several studies on functional cost parameters are needed to achieve the best trajectory speed of the robotic arm. These studies could attempt to optimise the functional cost parameters and provide a mechanical characterisation of brain tissue based on real data